BPI-GNN: Interpretable brain network-based psychiatric diagnosis and subtyping.

Converging evidence increasingly suggests that psychiatric disorders, such as major depressive disorder (MDD) and autism spectrum disorder (ASD), are not unitary diseases, but rather heterogeneous syndromes that involve diverse, co-occurring symptoms and divergent responses to treatment. This clinical heterogeneity has hindered the progress of precision diagnosis and treatment effectiveness in psychiatric disorders. In this study, we propose BPI-GNN, a new interpretable graph neural network (GNN) framework for analyzing functional magnetic resonance images (fMRI), by leveraging the famed prototype learning. In addition, we introduce a novel generation process of prototype subgraph to discover essential edges of distinct prototypes and employ total correlation (TC) to ensure the independence of distinct prototype subgraph patterns. BPI-GNN can effectively discriminate psychiatric patients and healthy controls (HC), and identify biological meaningful subtypes of psychiatric disorders. We evaluate the performance of BPI-GNN against 11 popular brain network classification methods on three psychiatric datasets and observe that our BPI-GNN always achieves the highest diagnosis accuracy. More importantly, we examine differences in clinical symptom profiles and gene expression profiles among identified subtypes and observe that our identified brain-based subtypes have the clinical relevance. It also discovers the subtype biomarkers that align with current neuro-scientific knowledge.

CI-GNN: A Granger causality-inspired graph neural network for interpretable brain network-based psychiatric diagnosis.

There is a recent trend to leverage the power of graph neural networks (GNNs) for brain-network based psychiatric diagnosis, which, in turn, also motivates an urgent need for psychiatrists to fully understand the decision behavior of the used GNNs. However, most of the existing GNN explainers are either post-hoc in which another interpretive model needs to be created to explain a well-trained GNN, or do not consider the causal relationship between the extracted explanation and the decision, such that the explanation itself contains spurious correlations and suffers from weak faithfulness. In this work, we propose a granger causality-inspired graph neural network (CI-GNN), a built-in interpretable model that is able to identify the most influential subgraph (i.e., functional connectivity within brain regions) that is causally related to the decision (e.g., major depressive disorder patients or healthy controls), without the training of an auxillary interpretive network. CI-GNN learns disentangled subgraph-level representations α and ß that encode, respectively, the causal and non-causal aspects of original graph under a graph variational autoencoder framework, regularized by a conditional mutual information (CMI) constraint. We theoretically justify the validity of the CMI regulation in capturing the causal relationship. We also empirically evaluate the performance of CI-GNN against three baseline GNNs and four state-of-the-art GNN explainers on synthetic data and three large-scale brain disease datasets. We observe that CI-GNN achieves the best performance in a wide range of metrics and provides more reliable and concise explanations which have clinical evidence. The source code and implementation details of CI-GNN are freely available at GitHub repository (https://github.com/ZKZ-Brain/CI-GNN/).

Methionine-CBS axis promotes intracellular ROS levels by reprogramming serine metabolism.

As a non-essential amino acid, cysteine could be obtained through both exogenous uptake and endogenous de novo synthesis pathways. Research has demonstrated that restricting the uptake of cystine could result in a depletion of intracellular cysteine and glutathione, ultimately leading to an increase in intracellular reactive oxygen species (ROS) levels. However, the role of methionine in regulating intracellular ROS levels is currently unclear. Here, we want to explore the role of methionine in regulating intracellular ROS levels. We found that methionine restriction could lead to a decrease in intracellular ROS levels, while supplementation with SAM can restore these levels through flow cytometry. Mechanically, we found that the methionine-SAM axis relies on CBS when regulating intracellular ROS levels. Furthermore, we speculate and prove that the methionine-SAM-CBS axis alters the metabolism of serine, thereby reducing intracellular reductive power, therefore promoting intracellular ROS levels through changing metabolite levels and genetic methods. Finally, our study revealed that high expression of CBS in tumor cells could lead to increased intracellular ROS levels, ultimately resulting in faster proliferation rates. Together, our study confirmed that methionine plays a promoting role in the regulation of intracellular ROS levels.

Analysis of Deep Convolutional Neural Networks Using Tensor Kernels and Matrix-Based Entropy.

Analyzing deep neural networks (DNNs) via information plane (IP) theory has gained tremendous attention recently to gain insight into, among others, DNNs' generalization ability. However, it is by no means obvious how to estimate the mutual information (MI) between each hidden layer and the input/desired output to construct the IP. For instance, hidden layers with many neurons require MI estimators with robustness toward the high dimensionality associated with such layers. MI estimators should also be able to handle convolutional layers while at the same time being computationally tractable to scale to large networks. Existing IP methods have not been able to study truly deep convolutional neural networks (CNNs). We propose an IP analysis using the new matrix-based Rényi's entropy coupled with tensor kernels, leveraging the power of kernel methods to represent properties of the probability distribution independently of the dimensionality of the data. Our results shed new light on previous studies concerning small-scale DNNs using a completely new approach. We provide a comprehensive IP analysis of large-scale CNNs, investigating the different training phases and providing new insights into the training dynamics of large-scale neural networks.

A Componentwise Approach to Weakly Supervised Semantic Segmentation Using Dual-Feedback Network.

Recent weakly supervised semantic segmentation methods generate pseudolabels to recover the lost position information in weak labels for training the segmentation network. Unfortunately, those pseudolabels often contain mislabeled regions and inaccurate boundaries due to the incomplete recovery of position information. It turns out that the result of semantic segmentation becomes determinate to a certain degree. In this article, we decompose the position information into two components: high-level semantic information and low-level physical information, and develop a componentwise approach to recover each component independently. Specifically, we propose a simple yet effective pseudolabels updating mechanism to iteratively correct mislabeled regions inside objects to precisely refine high-level semantic information. To reconstruct low-level physical information, we utilize a customized superpixel-based random walk mechanism to trim the boundaries. Finally, we design a novel network architecture, namely, a dual-feedback network (DFN), to integrate the two mechanisms into a unified model. Experiments on benchmark datasets show that DFN outperforms the existing state-of-the-art methods in terms of intersection-over-union (mIoU).

Functional Connectome of the Human Brain with Total Correlation.

Recent studies proposed the use of Total Correlation to describe functional connectivity among brain regions as a multivariate alternative to conventional pairwise measures such as correlation or mutual information. In this work, we build on this idea to infer a large-scale (whole-brain) connectivity network based on Total Correlation and show the possibility of using this kind of network as biomarkers of brain alterations. In particular, this work uses Correlation Explanation (CorEx) to estimate Total Correlation. First, we prove that CorEx estimates of Total Correlation and clustering results are trustable compared to ground truth values. Second, the inferred large-scale connectivity network extracted from the more extensive open fMRI datasets is consistent with existing neuroscience studies, but, interestingly, can estimate additional relations beyond pairwise regions. And finally, we show how the connectivity graphs based on Total Correlation can also be an effective tool to aid in the discovery of brain diseases.

Modularizing Deep Learning via Pairwise Learning With Kernels.

By redefining the conventional notions of layers, we present an alternative view on finitely wide, fully trainable deep neural networks as stacked linear models in feature spaces, leading to a kernel machine interpretation. Based on this construction, we then propose a provably optimal modular learning framework for classification that does not require between-module backpropagation. This modular approach brings new insights into the label requirement of deep learning (DL). It leverages only implicit pairwise labels (weak supervision) when learning the hidden modules. When training the output module, on the other hand, it requires full supervision but achieves high label efficiency, needing as few as ten randomly selected labeled examples (one from each class) to achieve 94.88% accuracy on CIFAR-10 using a ResNet-18 backbone. Moreover, modular training enables fully modularized DL workflows, which then simplify the design and implementation of pipelines and improve the maintainability and reusability of models. To showcase the advantages of such a modularized workflow, we describe a simple yet reliable method for estimating reusability of pretrained modules as well as task transferability in a transfer learning setting. At practically no computation overhead, it precisely described the task space structure of 15 binary classification tasks from CIFAR-10.

The efficacy of acoustic therapy versus oral medication for chronic tinnitus: A meta-analysis.

PURPOSE: To compare the efficacy of acoustic therapy (AT) and drug therapy (DT) for chronic tinnitus. METHODS: We searched Pubmed, ScienceDirect, Chinese Journal Full-text Database (CNKI), Wanfang Database, Chinese Biomedical Literature Database (CBM), Embase, and Cochrane Library from the establishment of the database to December 2019. Meta-analysis was performed on the Tinnitus Handicap Inventory (THI) score and Visual Analogue Scale (VAS) with included literature using Revman 5.3 software. RESULTS: A total of 18 documents were included, including 16 Chinese documents and 2 English documents, with 1774 patients (including 962 patients treated with AT and 812 patients treated with DT). The effect of AT (by the number of cases or ears) is better than that of DT (P < 0.05). After treatment, the THI value of AT was more evident than that of DT (WMD = -4.25, (-13.24, -5.29)). And the VAS value of AT was significantly lower than that of DT (WMD = -0.73, (-1.31, -0.15)). CONCLUSION: Compared with DT, AT can significantly improve the efficacy of tinnitus and reduce the symptoms of tinnitus patients. Clinically, it can vigorously promote the application value of treating tinnitus by sound.

LINC01303 promotes the proliferation and migration of laryngeal carcinoma by regulating miR-200c/TIMP2 axis.

BACKGROUND: It is reported that long non-coding RNA is crucial in many cancer progressions. But the function and regulatory mechanism of LINC01303 in human laryngeal squamous cell carcinoma (LSCC) remains unclear. Hence, this research aims at investigating the biological function and potential mechanism of LINC01303 in LSCC. METHODS: Real-time quantitative PCR (qRT-PCR) was applied for the determination of LINC01303, miR-200c and TIMP metallopeptidase inhibitor 2 (TIMP2) expression in LSCC tissues and cell lines. Corresponding experiments were carried out to determine the impacts of LINC01303 on LSCC cell proliferation, apoptosis, migration and invasion. The interaction between LINC01303 and miR-200c was analyzed with bioinformatics analysis and luciferase activity analysis. RESULTS: LINC01303 expression in LSCC tissues was notably higher than that in adjacent normal tissues. High LINC01303 expression was bound up with lymphatic metastasis and advanced clinical stage. In addition, inhibition of LINC01303 by siRNA could evidently block LSCC cell proliferation, induce apoptosis, and inhibit invasion and migration. Mechanically, LINC01303 acted as carcinogenic lncRNA in LSCC by regulating miR-200c/TIMP2 axis. CONCLUSION: LINC01303 plays a carcinogenic part in LSCC carcinogenesis through regulating miR-200c/TIMP2 axis, which may become a promising target of LSCC therapy.

Understanding Convolutional Neural Networks With Information Theory: An Initial Exploration.

A novel functional estimator for Rényi's α -entropy and its multivariate extension was recently proposed in terms of the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the utility and possible applications of these new estimators are rather new and mostly unknown to practitioners. In this brief, we first show that this estimator enables straightforward measurement of information flow in realistic convolutional neural networks (CNNs) without any approximation. Then, we introduce the partial information decomposition (PID) framework and develop three quantities to analyze the synergy and redundancy in convolutional layer representations. Our results validate two fundamental data processing inequalities and reveal more inner properties concerning CNN training.

Reconstruction of giant full-thickness lower eyelid defects using a combination of palmaris longus tendon with superiorly based nasolabial skin flap and palatal mucosal graft.

Reconstruction of a full-thickness lower eyelid defect is challenging. We aim to use palmaris longus tendon to improve clinical outcomes in eyelid reconstruction. We generated a novel "three-layer structure" tissue by combination of palmaris longus tendon with superiorly-based nasolabial skin flap and palatal mucosal graft and applied in eyelid reconstruction surgery in 34 patients with significant full-thickness lower eyelid defects. The satisfaction scores were assessed in each patient to evaluate their cosmetic and functional outcomes in follow-up visits. The mean follow-up period was 15 months (range, 6-24 months). Satisfactory results were obtained in 100% patients. No patients reported deformities, obvious scars at the donor sites, or abnormalities of hand function on the surgical side. Our results demonstrated that the three-layer structure incorporating palmaris longus tendon for the reconstruction of giant full-thickness defects in lower eyelid is an effective procedure with satisfactory long-term results.

Multiview Hybrid Embedding: A Divide-and-Conquer Approach.

We present a novel cross-view classification algorithm where the gallery and probe data come from different views. A popular approach to tackle this problem is the multiview subspace learning (MvSL) that aims to learn a latent subspace shared by multiview data. Despite promising results obtained on some applications, the performance of existing methods deteriorates dramatically when the multiview data is sampled from nonlinear manifolds or suffers from heavy outliers. To circumvent this drawback, motivated by the Divide-and-Conquer strategy, we propose multiview hybrid embedding (MvHE), a unique method of dividing the problem of cross-view classification into three subproblems and building one model for each subproblem. Specifically, the first model is designed to remove view discrepancy, whereas the second and third models attempt to discover the intrinsic nonlinear structure and to increase the discriminability in intraview and interview samples, respectively. The kernel extension is conducted to further boost the representation power of MvHE. Extensive experiments are conducted on four benchmark datasets. Our methods demonstrate the overwhelming advantages against the state-of-the-art MvSL-based cross-view classification approaches in terms of classification accuracy and robustness.

Multivariate Extension of Matrix-Based Rényi's α-Order Entropy Functional.

The matrix-based Rényi's α-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the current theory in the matrix-based Rényi's α-order entropy functional only defines the entropy of a single variable or mutual information between two random variables. In information theory and machine learning communities, one is also frequently interested in multivariate information quantities, such as the multivariate joint entropy and different interactive quantities among multiple variables. In this paper, we first define the matrix-based Rényi's α-order joint entropy among multiple variables. We then show how this definition can ease the estimation of various information quantities that measure the interactions among multiple variables, such as interactive information and total correlation. We finally present an application to feature selection to show how our definition provides a simple yet powerful way to estimate a widely-acknowledged intractable quantity from data. A real example on hyperspectral image (HSI) band selection is also provided.

On Kernel Method-Based Connectionist Models and Supervised Deep Learning Without Backpropagation.

We propose a novel family of connectionist models based on kernel machines and consider the problem of learning layer by layer a compositional hypothesis class (i.e., a feedforward, multilayer architecture) in a supervised setting. In terms of the models, we present a principled method to "kernelize" (partly or completely) any neural network (NN). With this method, we obtain a counterpart of any given NN that is powered by kernel machines instead of neurons. In terms of learning, when learning a feedforward deep architecture in a supervised setting, one needs to train all the components simultaneously using backpropagation (BP) since there are no explicit targets for the hidden layers (Rumelhart, Hinton, & Williams, 1986). We consider without loss of generality the two-layer case and present a general framework that explicitly characterizes a target for the hidden layer that is optimal for minimizing the objective function of the network. This characterization then makes possible a purely greedy training scheme that learns one layer at a time, starting from the input layer. We provide instantiations of the abstract framework under certain architectures and objective functions. Based on these instantiations, we present a layer-wise training algorithm for an l-layer feedforward network for classification, where l≥2 can be arbitrary. This algorithm can be given an intuitive geometric interpretation that makes the learning dynamics transparent. Empirical results are provided to complement our theory. We show that the kernelized networks, trained layer-wise, compare favorably with classical kernel machines as well as other connectionist models trained by BP. We also visualize the inner workings of the greedy kernelized models to validate our claim on the transparency of the layer-wise algorithm.

Understanding autoencoders with information theoretic concepts.

Despite their great success in practical applications, there is still a lack of theoretical and systematic methods to analyze deep neural networks. In this paper, we illustrate an advanced information theoretic methodology to understand the dynamics of learning and the design of autoencoders, a special type of deep learning architectures that resembles a communication channel. By generalizing the information plane to any cost function, and inspecting the roles and dynamics of different layers using layer-wise information quantities, we emphasize the role that mutual information plays in quantifying learning from data. We further suggest and also experimentally validate, for mean square error training, three fundamental properties regarding the layer-wise flow of information and intrinsic dimensionality of the bottleneck layer, using respectively the data processing inequality and the identification of a bifurcation point in the information plane that is controlled by the given data. Our observations have direct impact on the optimal design of autoencoders, the design of alternative feedforward training methods, and even in the problem of generalization.

Simple Stopping Criteria for Information Theoretic Feature Selection.

Feature selection aims to select the smallest feature subset that yields the minimum generalization error. In the rich literature in feature selection, information theory-based approaches seek a subset of features such that the mutual information between the selected features and the class labels is maximized. Despite the simplicity of this objective, there still remain several open problems in optimization. These include, for example, the automatic determination of the optimal subset size (i.e., the number of features) or a stopping criterion if the greedy searching strategy is adopted. In this paper, we suggest two stopping criteria by just monitoring the conditional mutual information (CMI) among groups of variables. Using the recently developed multivariate matrix-based Rényi's α-entropy functional, which can be directly estimated from data samples, we showed that the CMI among groups of variables can be easily computed without any decomposition or approximation, hence making our criteria easy to implement and seamlessly integrated into any existing information theoretic feature selection methods with a greedy search strategy.

Fast sparse fractal image compression.

As a structure-based image compression technology, fractal image compression (FIC) has been applied not only in image coding but also in many important image processing algorithms. However, two main bottlenecks restrained the develop and application of FIC for a long time. First, the encoding phase of FIC is time-consuming. Second, the quality of the reconstructed images for some images which have low structure-similarity is usually unacceptable. Based on the absolute value of Pearson's correlation coefficient (APCC), we had proposed an accelerating method to significantly speed up the encoding of FIC. In this paper, we make use of the sparse searching strategy to greatly improve the quality of the reconstructed images in FIC. We call it the sparse fractal image compression (SFIC). Furthermore, we combine both the APCC-based accelerating method and the sparse searching strategy to propose the fast sparse fractal image compression (FSFIC), which can effectively improve the two main bottlenecks of FIC. The experimental results show that the proposed algorithm greatly improves both the efficiency and effectiveness of FIC.